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a  ✓'  _vV  uality  is  never  an  accident.  It 
I  1  is  always  the  result  of  intel- 
ligent  effort.”  John  Ruskin 

As  the  national  and  international  de¬ 
bate  on  progress  in  Afghanistan  contin¬ 
ues,  many  pundits  are  now  turning  their 
attention  to  the  quality  of  the  Afghan 
National  Security  Forces  (ANSF)  we  are 
producing  here  shoulder-to-shoulder 
with  our  Afghan  counterparts  rather  than 
the  quantity.  This  is  a  logical  follow-on 
argument  given  the  Afghan’s  displayed 
ability  to  achieve  their  2010  growth  goals 
early  and  their  continued  positive  growth 
trends.  But,  what  does  quality  really  mean 
and  how  do  we  measure  it?  Unfortunate¬ 
ly,  there  is  no  easy  answer  to  this  question 
and  critics  everywhere  will  likely  grasp  at 
convenient,  but  potentially  misleading, 
measures  and  use  them  to  advance  their 
argument.  Having  examined  the  topic  of 
measuring  the  quality  of  Afghan  forces  in 
detail  with  some  very  capable  colleagues, 
we  offer  this  essay  as  a  point  of  departure 
for  the  debate  on  quality.  Our  intention 
is  not  to  provide  empirical  evidence  that 
the  Afghan  Security  Forces  are  “quality” 
forces — they  are — but  rather  to  show  the 
limitations  and  difficulties  to  the  use  of 
quality  as  a  metric.  We  hope  to  learn  from 
the  ensuing  debate  and  use  those  lessons 
to  refine  our  current  processes. 

The  ultimate  measure  of  quality  for 
any  security  force  is  operational  effective¬ 
ness — can  the  force  successfully  execute 
its  mission  on  the  battlefield?  The  Inter¬ 
national  Security  Assistance  Force  (ISAF) 
Joint  Command  employed  through 
May  2010  a  largely  objective  system  of 
measures  called  the  Training  and  Readi¬ 
ness  Assessment  Tool  (TRAT)  process  to 
measure  unit  effectiveness.  Following  the 
Special  Inspector  General  for  Afghani¬ 
stan  Reconstruction  (SIGAR)  report, 
the  Commander’s  Unit  Assessment  Tool 
(CUAT)  replaced  TRAT  and  added  a 
subjective  portion  to  the  assessment  in  an 
effort  to  better  articulate  true  capability 
of  the  unit  to  function  on  the  battlefield. 
While  many  believe  the  CUAT  is  an 
improvement  over  the  TRAT,  two  linger¬ 
ing  issues  prevent  its  wide  acceptance  as 
a  true  measure  of  quality.  First,  the  ISAF 
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Joint  Command  (IJC)  continues  to  tweak 
the  system,  which  has  yet  to  produce 
steady-state  results  and  therefore  limits 
any  trend  analysis.  Second,  many  are  un¬ 
willing  to  accept  a  single  metric  like  the 
CUAT-produced  Rating  Definition  Level 
(RDL)  as  a  measure  of  quality.  Perhaps 
it  is  unfamiliarity  with  the  process  or  the 
lack  of  transparency  that  prevents  wider 
acceptance  of  the  RDL.  While  we  believe 
the  CUAT  is  the  right  system  and  the 
RDL  is  the  correct  metric  to  assess  overall 
operational  quality  of  the  force,  we  also 
believe  that  NATO  Training  Mission — 
Afghanistan  (NTM-A)  produces  quality 
outputs  from  the  institutional  base  that 
contribute  to  the  overall  effectiveness  of 
Afghan  units.  The  difficulty  is  measuring 
this  quality. 

Before  we  try  to  articulate  how  we 
think  quality  should  be  viewed,  we  must 
state  an  undeniable  fact.  Quality  is  inher¬ 
ent  in  everything  we  do.  We  never  set 
out  to  produce  anything  less — a  fact  we 
think  is  lost,  or  at  least  not  acknowledged 
by  many  critics  and  commentators.  We 
are  not  wired  to  do  otherwise.  All  of  our 
actions  are  an  effort  to  maximize  qual¬ 
ity  subject  to  the  constraints  we  face. 

The  constraints  we  face  are  composed  of 
those  placed  upon  us  by  our  civilian  and 
uniformed  leaders  and  those  driven  by 
the  environment  we  operate  in  here  in  Af¬ 
ghanistan.  We  are  constrained  by  the  time 
given  to  accomplish  our  tasks  or  mission 
and  by  the  needs  of  the  operational  com¬ 
mander,  which  place  time  limits  on  our 
institutional  processes.  Funding  is  finite 
and  the  controls  placed  on  these  resources 
constrain  our  actions,  as  do  our  own 
efforts  to  remain  good  stewards  of  these 
resources.  Manpower  further  constrains 
our  actions  and  ability  to  produce  quality 
and  will  only  become  more  restrictive  as 
we  shift  focus  to  the  fielding  of  techni¬ 
cally  intensive  specialties.  Finally,  the  en¬ 
vironment  of  Afghanistan  has  the  greatest 
constraint  on  our  quest  for  quality.  The 
lack  of  infrastructure,  the  pervasiveness  of 
corruption,  and  the  largely  illiterate  popu¬ 
lation  among  many  other  environmental 
factors  make  the  attainment  of  quality 
that  much  more  difficult  to  achieve.  Dif¬ 
ficult — yes,  impossible — no;  we  prove  it 
every  day. 
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Although  operations  are  the  respon¬ 
sibility  of  IJC,  everything  we  do  here  in 
NTM-A  directly  contributes  to  the  opera¬ 
tional  effectiveness  of  the  ANSF.  Whether 
we  are  building  a  facility,  advising  an 
Afghan  counterpart  on  policy  within  one 
of  the  security  ministries,  or  training  the 
next  Afghan  Kandak  to  enter  battlespace, 
we  are  contributing  to  the  quality  of 
the  ANSF.  Therefore,  the  quality  of  our 
outputs  is  nested  and  contributes  to  the 
overall  quality  that  is  measured  through 
the  CUAT.  However,  unlike  the  CUAT, 
our  processes  and  outputs  are  so  diverse 
that  a  single  measure  of  quality  is  indefin¬ 
able.  Naturally,  we  collect  many  metrics 
that  are  indicators  of  quality,  but  no 
single  metric  or  mathematical  aggregation 
of  metrics  is  satisfying,  let  alone  feasible 
to  be  a  standalone  measure  of  quality.  The 
best  we  can  do  is  to  develop  an  over¬ 
arching  qualitative  narrative  assessment 
supported  by  a  series  of  quantitative  and 
qualitative  indicators  that  describe  the 
quality  of  our  outputs. 

Several  issues  with  assessing  quality 
must  be  discussed  before  we  describe  the 
framework  of  this  narrative.  First,  when 
assessing  quality  we  should  avoid  com¬ 
parative  assessments  that  are  not  instruc¬ 
tive.  For  example,  to  assess  quality  by 
comparing  an  Afghan  noncommissioned 
officer  (NCO)  to  a  coalition  NCO  is  not 
useful  because  the  systemic  differences  are 
so  great  between  the  two.  We  have  just 
started  to  create  an  NCO  corps  in  the  Af¬ 
ghan  Army  while  coalition  NCOs  are  the 
result  of  a  system  that  has  matured  over 
centuries.  This  supports  the  second  point, 
where  assessments  of  quality  should  be 
objective  and  not  subjective.  More  specifi¬ 
cally,  the  objective  assessment  must  be 
within  an  Afghan  context,  not  a  Western 
one.  A  true  assessment  of  quality  must 
be  one  that  is  objective  and  measures  the 
current  state  against  the  planned  state  en¬ 
visioned  for  this  period  of  time.  We  must 
account  for  this  temporal  aspect  when 
assessing  quality.  There  are  certain  at¬ 
tributes  that  are  required  today  to  support 
immediate  operational  needs,  but  others 
will  be  required  in  the  future  as  the  ANSF 
transforms  to  a  National  Defense/Rule  of 
Law  based  force.  It  is  not  instructive  to 
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measure  the  quality  of  today’s  ANSF  with 
the  same  yardstick  used  to  measure  a  force 
with  centuries  of  institutional  history. 

The  framework  we  propose  for  the  nar¬ 
rative  assessment  of  quality  of  the  ANSF 
is  composed  of  four  pillars — education, 
training,  experience,  and  effectiveness. 
These  four  pillars  are  fundamental  ele¬ 
ments  of  any  operationally  effective  force. 
These  are  the  same  elements  we  use  in 
the  U.S.  Army  to  life-cycle  manage  not 
only  our  soldiers,  NCOs,  and  officers,  but 
also  our  units,  to  a  certain  extent.  Each 
element  is  supported  with  qualitative  and / 
or  quantitative  data  that  the  narrative  is 
based  upon.  We  will  briefly  describe  each 
of  the  elements  in  greater  detail. 

Education  is  an  essential  component 
of  the  development  of  all  military  leaders. 
Military  and  civilian  education  develops 
the  attributes  and  traits  that  are  required 
by  effective  leaders.  It  serves  as  the  core  of 
the  professional  development  of  our  lead¬ 
ers  and  is  a  career-long  endeavor.  Within 
the  context  of  education,  quality  is  gained 
by  exposing  the  individual  soldier  to 
increasing  levels  of  educational  opportu¬ 
nities.  In  a  nation  where  the  basic  literacy 
rate  is  estimated  at  28%,  literacy  and 
numeracy  rates  are  key  measures  of  qual¬ 
ity.  The  greater  percentage  of  the  ANSF 
that  we  can  make  increasingly  literate  will 
directly  benefit  operational  effectiveness 
(e.g.,  orders  and  directives)  and  steward¬ 
ship  of  personnel  and  equipment  (e.g., 
supply  accountability).  More  importantly 
is  the  second-  and  third-order  effects  a 
more  literate  society  has  on  the  gover¬ 
nance  and  development  of  the  country. 

It  is  basic  literacy  that  will  serve  as  the 
building  block  to  professionalization  of 
the  force.  Measuring  quality  here  includes 
the  number  of  Professional  Military  Edu¬ 
cation  (PME)  courses  established  and  the 
number  of  graduates  from  these  courses. 
In  assessing  graduation  rates  of  PME 
courses,  it  is  important  to  remember  that 
operations  limit  the  potential  number  of 
attendees.  Simply  counting  PME  gradu¬ 
ates  is  not  sufficient  either,  so  a  required 
distribution  of  graduates  throughout  the 
ANSF  should  be  developed  and  qual¬ 
ity  assessed  as  they  progress  toward  this 
goal.  The  next  two  pillars  build  upon  this 
foundation  of  education. 

Training  is  designed  to  prepare  sol¬ 
diers,  policemen,  NCOs,  and  officers 
to  fight  and  win  in  battle.  NTM-A  is 
charged  with  the  mission  of  training  the 
ANSF,  creating  the  institutional  training 


base,  and  developing  Afghan  capacity  to 
operate  the  training  base  independently. 
This  is  analogous  to  training  someone  to 
fly  an  airplane  as  it  is  being  built  around 
them.  This  has  been,  and  will  continue 
to  be,  a  challenging  mission,  but  it  is  an 
area  with  numerous  metrics  that  directly 
(and  indirectly)  measure  quality.  Quality 
commitment  is  evident  in  the  process  for 
modifying  Programs  of  Instruction  (POI) 
based  on  lessons  learned  and  instructed 
by  Afghan  combat  veterans.  The  number 
of  Afghan  certified  trainers  measures  the 
quality  of  instruction  because  Afghans 
(as  we  have  learned)  learn  best  from  their 
peers.  The  number  of  branch  school 
courses  and  graduates  from  these  courses 
is  indicative  of  quality  in  that  it  shows 
the  extent  to  which  the  ANSF  is  provid¬ 
ing  its  own  enablers  and  leaders  of  these 
technically  challenging  fields.  Other 
useful  indicators  of  ANSF  quality  include 
marksmanship  rates,  instructor  to  student 
ratios,  and  collective  training  validation 
scores.  Obviously  this  is  not  an  exhaustive 
list  of  relevant  measures;  however  these 
are  easily  captured  and  are  applicable  for 
describing  quality  of  the  ANSF.  We  will 
now  look  at  the  third  pillar — experience, 
which  more  than  any  other  requires  time 
to  generate  true  quality. 

Experience  is  invaluable  for  security 
forces,  especially  for  leaders,  and  cannot 
be  replicated  or  replaced  by  any  amount 
of  training  and  education.  Experience 
cannot  be  accelerated  either;  it  must  be 
earned  over  time  and  then  the  appropriate 
lessons  must  be  applied  and  reinforced  so 
that  they  become  inculcated  into  the  fiber 
of  the  unit  or  individual.  The  respon¬ 
sibility  of  ISAF  is  to  ensure  the  ANSF 
gains  experience  at  the  appropriate  rate 
and  under  the  appropriate  conditions. 
Experience-derived  metrics  on  quality  are 
focused  on  measuring  the  opportunities 
for  soldiers  and  police  to  gain  experience. 
Specifically,  the  average  time  of  service 
is  a  great  indicator  of  quality  because 
the  longer  an  Afghan  serves  the  greater 
the  experience  base  and  the  greater  the 
effectiveness.  In  the  absence  of  a  by-name 
personnel  system  it  is  impossible  to  track 
average  time  in  service.  We  can,  however, 
consider  metrics  such  as  retention,  attri¬ 
tion  and  absent-without-leave  (AWOL) 
rates,  and  NCO  and  officer  fill  rates  as 
an  indicator  of  increasing  average  time 
in  service  and  consequently  increasing 
experience.  Additionally,  we  can  assist  the 
ministries  in  implementing  initiatives  that 


incentivize  longer  or  continued  service 
beyond  initial  terms  such  as  merit-based 
promotions,  award  systems  that  recognize 
performance  and  achievement,  and  a  re¬ 
tirement  system  that  enables  management 
of  the  force  and  expands  opportunities 
for  experience.  Because  we  are  working 
toward  ANSF  independent  operations, 
we  must  also  look  at  measures  of  minis¬ 
terial  and  enduring  institution  capacity 
as  indicators  of  quality.  Capacity  gained 
in  these  two  areas  is  largely  experien- 
tially  driven  and  allows  for  the  ANSF  to 
become  self-sufficient  from  the  national 
to  local  levels.  The  more  the  ANSF  can  do 
for  itself,  the  higher  quality  of  the  force. 
Finally,  partnering  is  critical  to  ensuring 
that  experience  is  gained  under  optimal 
conditions.  Partnering  coverage  and  units 
with  an  operational  deployment  cycle  are 
measures  of  quality  of  experienced  gained. 
Experience  built  upon  a  foundation  of 
education  and  training  contributes  to  the 
effectiveness  of  individuals  and  institu¬ 
tions,  the  last  pillar  we  will  discuss. 

Operational  effectiveness,  the  per¬ 
formance  of  units  on  the  battlefield,  is 
supported  through  the  effectiveness  of 
individuals  and  institutions  from  the 
ministerial  to  the  tactical  level.  Quality 
indicators  are  derived  from  measures  of 
individual  and  institutional  effectiveness. 
Effective  individuals  operate  within  a 
code  of  conduct  (like  most  professions), 
are  not  corrupt,  and  selflessly  serve. 
Indicators  that  support  the  assessment 
of  effective  individuals  include  percent 
of  the  force  that  have  signed  a  code  of 
conduct  or  received  formal  training,  the 
number  of  corruption  convictions  or  inef¬ 
fective  leaders  removed  from  the  force, 
and  AWOL  rates  of  NCOs  and  officers. 
Institutional  effectiveness  is  indicated  by 
stewardship  of  resources  and  systemic 
linkages  from  national  to  local  levels. 
Institutional  capacity  and  Operational 
Deployment  Cycles  were  presented  in  the 
previous  discussion  on  experience,  but 
are  applicable  here  as  well.  Metrics  that 
measure  stewardship  include  equipment 
accountability  and  serviceability  of  equip¬ 
ment  and  facilities.  Greater  accountability 
and  serviceability  rates  indicate  increased 
effectiveness  and  thus  higher  quality. 
Systemic  performance  (Logistics,  Train¬ 
ing,  etc.)  is  more  difficult  to  quantify, 
but  would  measure  the  effectiveness  of 
translating  policy  into  critical  outputs  and 
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meeting  the  needs  of  the  force.  Increased 
effectiveness  of  these  systems  indicates  in¬ 
creased  quality  and  will  ultimately  lead  to 
greater  self-sufficiency  and  sustainability. 

The  opening  quotation  attributed  to 
John  Ruskin  truly  describes  our  quest  for 
quality  in  NTM-A.  Quality  is  inherent  in 
everything  we  do.  Unfortunately,  many 
don’t  realize  this  statement  of  fact  and  do 
not  account  for  it  in  their  calculus  when 
assessing  the  quality  of  the  ANSF.  Our 
challenge  is  not  building  a  quality  force, 
but  rather  how  we  measure  the  quality 
of  that  force  in  a  consistent,  comprehen¬ 
sive,  and  defendable  manner.  There  is 
no  single  quantifiable  measure  of  quality 
that  is  meaningful  and  meets  these  three 
criteria.  We  are  proposing  that  the  best 
way  to  describe  the  quality  of  the  ANSF  is 
to  adopt  an  overarching  narrative  assess¬ 
ment  that  weaves  together  the  series  of 
quantitative  and  qualitative  indicators  that 
directly  or  indirectly  measures  quality.  The 
narrative  should  be  based  on  a  framework 
of  education,  training,  experience,  and 


effectiveness;  realizing  there  are  others 
that  are  just  as  applicable.  To  ensure  the 
narrative  is  instructive  and  valid,  it  must  be 
objective,  conducted  through  an  Afghan 
perspective,  and  account  for  where  the 
ANSF  is  along  its  developmental  timeline. 
We  believe  the  methodology  presented 
here  meets  the  immediate  needs  of  our 
command,  but  could  benefit  from  ad¬ 
ditional  viewpoints  and  ideas — we  may  be 
too  close  to  the  problem.  We  welcome  the 
insights  of  others  and  look  forward  to  the 
debate. 
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The  security  environment  of  today 
includes  a  wide  range  of  “tar¬ 
gets”  that  must  be  tracked.  These 
include  potential  terrorists,  smugglers, 
those  illegally  accessing  natural  resources, 
paramilitary  naval  forces,  and  others,  on 
both  domestic  and  international  fronts. 
Dealing  with  this  security  environment 
requires  an  awareness  of  the  maritime 
domain;  often  referred  to  as  Maritime 
Domain  Awareness  (MDA)  or  Maritime 
Situational  Awareness  (MSA).  Generi- 
cally,  MDA  can  be  defined  as: 


“...  the  effective  understanding  of  everything 
on,  under,  related  to,  adjacent  to,  or 
bordering  a  sea,  ocean  or  other  navigable 
waterway,  including  all  maritime-related 
activities,  infrastructure,  people,  cargo, 
vessels,  or  other  conveyances.  ” 

September  2009,  the  Canadian  Assistant 
Deputy  Minister  Marine  Security  Committee 


In  October  2009,  under  the  advocacy 
of  the  U.S.  Northern  Command  (US¬ 
NORTHCOM),  the  Military  Operations 
Research  Society  (MORS)  and  Defence 
Research  and  Development  Canada,  Cen¬ 
tre  for  Operational  Research  and  Analysis 
(DRDC  CORA),  oversaw  a  meeting  to 
identify  and  discuss  MDA  issues  common 
to  the  United  States  and  Canada.  Con¬ 
ducted  at  the  Chateau  Cartier  in  Gatineau, 
Quebec,  more  than  115  attendees  from 
the  United  States,  Canada,  the  United 
Kingdom,  and  Norway  worked  to  tackle 
significant  analytic  problems  covering 
a  wide  range  of  issues  in  an  unclassified 
environment.  As  a  result  of  this  meeting, 
several  analytic  tools  were  developed  and 
fielded,  including  the  Maritime  Timeline 
Analysis  and  Requirements  Toolset  (M- 
TART),  as  described  in  the  article  by  Neil 
Carson  and  J.D.  Caron  of  the  same  name 


in  the  December  2010  issue  of  Phalanx. 

During  that  meeting,  the  organizing 
committee  was  approached  by  CAPT 
Kurt  Salchert,  CF,  and  CDR  Wayne 
Renaud,  CF.  Kurt  and  Wayne  suggested 
that  a  follow-on  meeting  be  convened 
within  1 8  months  designed  to  provide 
solutions  to  specific  operational  MDA 
problems,  posed  in  advance  of  the  work¬ 
shop,  that  are  of  interest  to  a  sponsoring 
agency  in  much  the  same  way  that  P.M.S 
Blackett  envisioned  when  he  penned  his 
now-famous  treatise  entitled  “Scientists 
at  the  Operational  Level.”1  A  proposal 
was  therefore  made  and  accepted  by  the 
MORS  sponsors  in  January  2010  that 
such  a  meeting  be  convened.  This  year 
MORS  and  DRDC  are  partnering  again 
to  conduct  our  second  MDA  meeting 
in  San  Diego  from  2—5  May  2011  at  the 
U.S.  Navy’s  Anti-Submarine  Warfare 
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